A trio of Princeton social scientists recently conducted a mass experiment with 160 research teams to see if any of them could predict how children's lives would turn out. The participants were given fifteen years of data and were allowed to use any technique they wanted, from good old fashioned statistical analysis to modern-day artificial intelligence. That's because artificial intelligence – much like psychics and headless chickens – cannot predict the future. Sure, it can predict trends and in some cases provide valuable insights that can help industries make the best decisions, but determining whether or not a child will become successful requires a level of prescience that brute-force mathematics can't provide. We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life outcomes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study.

A paper coauthored by over 112 researchers across 160 data and social science teams found that AI and statistical models, when used to predict six life outcomes for children, parents, and households, weren't very accurate even when trained on 13,000 data points from over 4,000 families. They assert that the work is a cautionary tale on the use of predictive modeling, especially in the criminal justice system and social support programs. "Here's a setting where we have hundreds of participants and a rich data set, and even the best AI results are still not accurate," said study co-lead author Matt Salganik, a professor of sociology at Princeton and interim director of the Center for Information Technology Policy at the Woodrow Wilson School of Public and International Affairs. "These results show us that machine learning isn't magic; there are clearly other factors at play when it comes to predicting the life course." The study, which was published this week in the journal Proceedings of the National Academy of Sciences, is the fruit of the Fragile Families Challenge, a multi-year collaboration that sought to recruit researchers to complete a predictive task by predicting the same outcomes using the same data.